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A review of the contri^but ions of evaluation to school 
television is presented, and soie policy suggestions for evaluation 
programs are offered. The purpose is to assist the Agency for 
Instructional Television (AIT) in deteriining how to utilize 
evaluative research in conducting its projects and the fdcus is upon 
decision-iakj ng related to the production of television programs. 
Four categories of decision-oriented research are 
identified — backgroundr foriative, suHBatiV€, and policy. Hajpr 
recoBBendai;ions Include: 1) Each iklT^ activity, should have a research 
and evaluation coiiponent; 2) AIT should focus upon decision-oriented 
and product-specific research evaluation; 3) Background and 

in-house policy research should be ^an integral part of AIT; 4) 
Foraative research should be conducted by an in-house staff and b\e 
organizationally^ subordinate ,to production; 5) Top priority for 
research resources should gb to foraative research; 6) Suaaative 
research, vhile necessary, is less iaportant than foraative research; 
7) Utility to the decision-aakiirg process is the chief criterion, by 
« hich«research and evaluation is to be judged; 8) Suaaative research 
for outside consuaption should be assigned to outside agencies; and 
9) Original instruaents should be used whenever possible for 
foraative and suaaative research. (Author/PB) 
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This POLICY BRIEF is provided to assist you in considering how 
the Agency for Instructional Television should employ research 
in its conduct .of cooperative program projects. After reviewing 
the brief, you will be asked to recommend a research policy for 
use by AIT, 

The AGENCY FOR INSTRUCTIONAL TELEVISION seeks to strengthen educa- 
tion in the United States and Canada, through television and other 
technologies. Its primary fuition is to expand and improve the 
cooperative develc^ment of major program projects. 

As an essential part of this function, AIT operates a COUNCIL FOR 
INSTRUCTIONAL TELEVISION. Th^ Council permits you and those with 
similar interests to consult regularly on matters of mutual con- 
cern, identify common needs, and make recommendations for imple- 
mentation by AITV 
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DFXISION-^TENTEO RESEARCH IN SCHOOL TELEVISION 



Section I : Background 

Introduction i " • 

^ ' - 

It takes a variety of talents and a series of coordinated decisions 
to get a television program successfully through the interlocking stages of 
planning, funding, production, evaluation, distribution, and utilization. 
This paper deals with the dzci6loyi6 involved in that process, and more - 
specifically, how research and evaluation can help /improve thq^se decisions. 

There are two primary objectives for this paper: 1} to review what 
evaluation can contribute to school television; and 2) to derive policy 
suggestions for such evaluation programs . S^' 

The scheme employed here uses four categories to make distinctions 
among several kinds^ of evaluation activities: 1) backgrouftd research; 
2) .formative research; 3) summative research; and 4) policy research. 
These categories will be defined, and their utility in decisionTmaking will 
be discussed. For each category, some dominant po^licy issues will be presented. 

I 

Focus on Decision-Making ^ 

There is no shortage of literature that examines and contrasts the 
procedures, assumptions, and purposes of basic research and evaluation (or ^ 
evaluative research).* These issues of similarity and contrast are quite 



the appended bibliography, see for example. Glass and VJorthen (1972o) 
Gut'E^ag (1971); Mielke (1973); Palmer (19720; and Phi Delta Kappa (^971). 



interesting, and debate continues on where ^'the various boundaries- should be 
located. Much of the "basic vs.. applied" research discission can be avoided 
here, however, if there is an early agreement on what the role of research and 
evaluation should be for t/ici organization and the projects it undertakes. 

A~^licy recommendation that can'serve to bypass mucli^ of tKe discussion 

s 

^nd debate on the. ' inctions between basic and applied research is: tlicit 
AIT dommU: iXb id^aaAck and avatuo^tion n.(Z^Oivicu o^xclMively to ddaJj^loyi- , . 
O'^ild^vtad and pn.oducX-6pQ,cl()lc -n^QAda^di, thus not engaginjkat -all in ih-house - 
research designed to test general theories or hypotheses. This places top 
priority on meeting, the specific decision-making needs of the organization and 
excludes from the research mission the objective^of making , a contribution' to 
new general izable knowledge. 

This should not be interpreted a^ a lack of interest in theoretical 
developments or a lack of recognition -of the eventual utility that AIT can 
derive from basic research. It is, however, a proper allocation of, resources 
for an- organization whose mission is not p^/i -52. academic or even scholarly, 
but mission-oriented and act;ion-oriented . School television acti^/'ities 

. ■ ■ • 

-^pursued oy AIT may offer enticing research opportunities to scholars both in 
and out of the organization. This may well generate a variety of overtures 
for full or partial sponsorship. The recommended attitudQ. here is one of 
support, encouragement,- and cooperation, but not as an official AIT activity, 
and not conducted under the financial sponsorship of AH . Under this policy, 
appropriate AIT research activities would 'generate data that one or more 

^ecision-makers should find useful, >^ - 



This orientation is entirely consistent with -the definition of evalua- 
tion proposed by the Phi Delta Kappa National Study Comittee on Hvaluat ion : 
'^Evaluation is the process of delineating, obtaining^ and providing useful ^ 
information for judging decision alternatives J" (p. xxv) This basic definition 
makes a clean break from the traditional inission of basic research, setting 
evaluation on a path of its own. . . ^ 



Distinctions among various types of decision - o ri ent ed research 

In principle, tlie domain of decision-oilented_^^C;f?3?ccit as' broad as v 
^.^^-^ r ^ 

the/domain of decisions that must be made. In complicated processes, such as 
developing, producing, and using televised ins trudlional materials, that domain 
is broad indeed. Within decision-oriented research several distinctions can he 
made and several categories defined. For example,' it may be useful to separate 
^valuations that should be, done by an in-hbuse tea^Ti of researchers from 
Revaluations that should be done on contract by independent researchers. In 
other cases, it may be useful to have chronologioai categories, such as pre- 
production, production and post-production research and evaluation. 

These category schemes or any other category schemes should be subject 
to the criterion of utility. When th'^y are not appropriate for an occasion, 
they should be modified to become useful, or they should be abandoned." It ds i 
hoped that this caveat will remove any sense of finality or reificatiori from the 
four-category scheme actually, employed here: 1) background research, 2) formative 
research, 3) suiamative research, and. 4) policy research. It will be argued 
that, background research and fgrmative research should be done by in-house 
personnel, while surnmative research and at least some kiYids of policy research 
should be done by outside, independent' organizations . Chronologically, 



background research is preproduction > formative research is done during the 
production period, while sununatlve research (amd probably policy research as 
well) would He conducted* after production has ended. 

Already implied in the development ^of the four- cat.egory array (back- 
ground, formative, summative, and ppl'.cy research) is the centrality of the 
television production proqess and the .Jiplevision procjuctj itself (i.e., the 

^.prograJii) . This is done in full recognition that the terminal objective is not 
to produce prograiiis^ but- to effect leamitig and behavior change. ' In chat small 
•sector of the total milieu scrutinized here, the television program, the 'stimulus, 
is the basic point df reference. The chains qf decisions around which the ' 

^various types of research" and evaluation will be developed in this papev are 
related diijectly to the planning, production, validation and utilization- 
of telpasion programs. The center of this small universe is television 

• Section IT. The Hour Categories - . ^ 
Background Research 

I£ we were* to run a survey on what' school Tadministrators instinctively 
think .of as' falling in the domain of instructional television -research and 
evaluation, it wouTd probably show that the activities cal lec\^'background 
research^' would not included. " A common notion is that researchers and 
evalUators come into the picture after the product is. complejied to evaluate 
the performance of that product. This is far too limiting, and indeed 
may not even be the most significant contribution that re^'earch and evalua- 
tion can make. yA cl^ear stand should be taken to recognize and create 
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oppjortvmities for making bettor decisions as a consequence of research and 
evaluation inputs . Long before producti-on is nompieted , indeed before pro- 
duction has even begun, systematic resea^^h is needed' for th? plethora of 
decisions that face the administraVor at the plnnning stage 

The generic, mission heie is tj reduce uncertainty in *the planning processv 
It seems appropriate to include within the category of background research at 
least such activiti.es as the assessment of needs tiiat can he addressed properly 

via television, the assessment of a host of audience or student variables, the 

I. 

assesstnent of the physical system for production^ distrilpution , and 'reception, 
and feasibility studie^ for fentati've program ideas. These are all proper 
subjects for research contribut^'ions that are not tied to any particular program 
or product.. (The many similarities to the processes of marketing research 
.should not go unreijognized . ) . ' 

It^ n.(icomnmnde.d that baakgA^ound ttvieo-tc/i co^AU^ng '^ick acLtivttXa^ cib 
tho^d 'tinted ai)Ov '^c inttgKatQd^pQAmmntly hvta thz ^tAuLdzvJii oi tko. on.gx\^^za^ 
tion, t^ po66.ibl^, KoXh^L thayi gta/iiyig up anol^' {^ox^aadv new 'pn.ojzcJ: each 
n(iw poteyvtlat ^^oun.cd {^undcng. A modest but sustained effort in this area ' 
should provide a reliable information source to administrative planners on 
such questions' as : 

. 1. What are the perceived curriculum needs? 

2. toat is the condition of the distribution svstemV 

3. Demographically , what is an accurate description of the potential 
audience? ' ' • • \ 

4. Psychologically, what is the audience attitude structure toward 
topics A, B, and C? . ' . 

5. »Vhat reception might we expect for a new series on s.ybject X? 



iJoz'ons of questions of this order can be submitted to ominrical treatments 
designed and pdmini stored by a team of researchers. , ■ 

It shoukl he affirmed here (although it applies equally to all cate 
tjqries of research and evaluation covered in this paper) that tlic data from 
researel^ and evaluation efforts do not pcY ie make the administrative decisions, 
nor do -they even ^ dictate a particular 'conclusion or interpretation, 'lliat is, 
research input'-is one source of information alongside several source's oi 
information and guidance that the administrator uses. With or without benefit 
of research input, the administrator must make decisions; the ^;qal Is to 
improve tho^e decisions through careful Iv . designed background rcs.earch. 

l-ormative Research - ' . , ' 

Formative research is the first of the four categories covered here to 
involve a specific program or project, and it should be ^initiated very early 
in the deveiopmentaT phase of that program. Formative research is typically 

defined in comparison with summative research, as research or evaluation ad- 

f 

ministered during the formative stages of a product {a television j^rograni in 

\ ; ■ ' ■ ' ' ■ . 

this case) that provides feedback to the production staff, enabling them to 
modify trnd improve the product before the final production decisions have 
been made. Formative research is pretesting programs early eno^j^h in the 
process to take corrective* action . Sunaiiative research, by way cf contrast, 
assesses the impact of a program' or series as a whole-, compariri^ob served ^ 
effects with the effects desired or anticipated (as stated in the behavioral 
goals and obj ectiveL) . Formative research is to summative research what re- 
peated midcourse guidance and corrective tutoring is to the final grade for 



the course. Seen in that light, the importance of the formative researc^v 
function should be readily apparent. 

Continuing the analog)^, one can speculate on the most rational decision 
in a. choice situation: if you could only have one or the other, either repeated 
midcourse guidance and corrective tutoring end-of -course evaluation' with 
final grade, which should you choose? Although one may rightly reject the 
impos T€r^ of such ^^either-or" conditions in reality, the questiorr is 
nevertheless instructive for the placement £>f priorities , and priorities 
are- required when the resources for research and evaluation are limited^ as 
they always are. The value position taken here is that once the agency 
has committed itself to a school television project, the top ^p^o^iXiJ ioK 
KUaan^dh ^kould go to (^onmattvn KUOjouicih, There should always 

be a suinji;ativi2 research component of some dimensions, but without an active 
formative research component, isolated summative research ^iiay be little more 
' til an an autopsy < 

As you. already know, production of teleivision programs is a complicated 
process in which innumerable assumptions about audience attention, compre- 
hension, and ot!,,— reactions must be made. Expertise iK,the production 
process is^needed.so that a fair share of these assumptions will turn out 
to be correct. Experience shows, however, that expert judgments alone (that 
is, expert predictions of audience react-ions] are flrequently^u)^OKiy, 
Ultimately, there is no substitute for a try-out of the program with repre- 
sentative audience members, 

\ 
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In forrhatdve x^bsearch, as in the otlier categories, the way to find out 
where the research effort yi>hoiitd b@ is, at least in Uirf^.o part, to work back 
from the decision-making needs of the person(s) to whom t'ae research re'^uits 
wilJ be addressed. This search has a logical and a human relations componenr. 
In formative research and evaluation, x.he consumer is the productibn staff and 
the decisions arc production decis'ions. The • production staff must be motivated 
to make use of the r^^search results; they must be able to undei:stand tlie research 

results;, and the research results must address the production decision$ appro- 

/ 

priate to that group of consumers.' Insights into the nature of producers and 
production can be used to generate guidelines (a) for motivational steps that 
can be taken, (b) for the form and purpose of research reports, and (c) for the^ 
type of questions for which data are sought. 

Consider first the motivational issue. Above all, there must be mutual 
respect, understanding, trust, and goodwill between research and production. 
Without dwel ling unduly on the theme of being --'nice" to o^^'^ another, suffice 
it to say that this social factor i^s probably far more critrical than any [ 
methodological or policy factor in dcterniijiing suc:ess or failure of a formative 
research effort. Researqhers should' understand that producers have ego invested 
^.n their creations, and it can he threatening to have the program criticized 
by the research staff. There is 'ittle hope of getting good data or influencing 
the improvcmpnt of productions if goodwill and mutual respect are not in 
evidence. . The essential climate of goodwill is quite fragile. 

The following practices, witnessed at Children's Television Workshop,, 
are offered here as policy recommendations for AIT a$ well: -1) To the max^mim: 
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ov'^ent possible, the consumer of formative .research should iiiculvcMl in '.euinj: 
the formative research agenda. In otht.'" words, ilu^ producers should 'r]Rn:ir;' 
what it is they need to knoWj and the ' rescar .hi: rj^, i thf^y can, shiuKl {jrovidu 
relevant data on- those issues,' (Note: This does jL- suggest tliat the research 
agenda should be restricted to those question : generated fwrn pr(;du«.:i ion ; : ' 
does state that real attpmpts should be made tu answc r respuns 1 1} 1 v an\' rvS'^sir Ji 
questioii posed by the produi^tion staff,) 

. 2) The fact that research is wording in the scrvic- of p ku^ti^m, and 
not vice versa, means that research .perforiris an advisory function: u iih tlic 
final decision-makdng power^ properly belonging to production, 

3) ^'There should be a continuing ■ in-house educative function j;oinj^ ^on 
between research and production, each enlightening the other about its .special 
areas of expertise and how the other party' c;ni best use tliat expertise, 

CoTisider now the research report. Given the reference to the needs of 
decision-makers, there are implications both for rlie content and the style 

of formative research reporting. The content must relate to the producer's - 

' ' ' ■ ■ - J- 

actuaZ decisions '(that is, must relate to -things within- the producer's power 
to ahang^) , This area of potential change is, of course, program design and 
production. Formative research should. speak to those still manipulable issues 
in program design and production, /i'mpljcations -for s.tyle ' should be governed b>' . 
the methodological sophistication of the consumerj the prodj^^ction staff. The 
jargon and statistical presentations appropriate for othJr researchers are 
usually inappropriate . for producers • • - 

The level of specificity in fvjrmative research reports, with the 
focus on specific production decisiors for specific programs; , 



r*' tends not to he vow rclcwint to thos ? not dirccllv involved m the nj-oJcct. lo 
make these report? i nte rest i n'l; to "outside" readers. woU 1 d require eons : de r.ih . t- 
reuritini^ and uould chani;e their Kuriction. A nolic>" r^^coi^^pioiula t i on is lIvMt. 
formative researcli repoi'ts retain tliei i' cM'^iMnal ]>urp(»se of sc'r\'inL: tlio- 
decision needs- ot' iirc^duct i on » aitd not he altered lo vAvci t.ho interests o}' oilier 
'readers. \n open inforna^ion i^olic}' reeonimeod{.'d {'k^v all resr^;irch i\']to it 
"hut on 1 >•' for ^ropotn s in>iieir oriiiinal forin» as [M'ejnired for their ori^^inal 
function. . ^ , 

^ . -Ai^" t- hites to'^i;ogi'ain design and product i <;^n , formative research also. 

■ h;as a role to.plJiV in the formulation and se 1 eel i oh? o f ohjpctives. (^ne pro- 
eedure is "for content e\]KM'ts to statcv j^oRjor;) 1 toriiis what the\- he 1 i e\ e" aix'^ 
fealistic i;oa 1 s> and./uhc^n for forma'f^Ve rese a rJ;!^ pc rsonne 1 ' to tVonsl ate ^these 

^aal's ift tQft nieasurah le o'hj c^ct i \es /-^ suh i ect to aj^pro\al h\' tlid or i i^i nwl 'content 

<fc * ' ' * ? / * ^ 

expert^. the ■ i 1 i t \" or inabilitv<to de\'ise acceptable beh a\'i o ra / 'measnres 

for 'j^weral'-goa is. may V'.cll he a si^^nificant criterion in'decidinci uhetlier or 
not to include that, "goal in the' mission foY the ])roi;rain. I!arl>" pre- test i n^i; 
on 'the achi cvabi i i t V of certain object i\es ' could also influence the desi gn 



of thCj^-j^o^i^ram. 

As vvi hi be J'i s cussed short 1 >' .yoj^erat i ona 1 1\- def i ncd obj ect i \'es are also 
nr. essent\^il part of tlie sunnative re.'scarcb oneration, where,, the interest i / 
not' in prcsi^ram desi qn and i niproxeir.ent , but. in' determining the degree Xif which 



the objectives were met. .Vlthoui^h fuf^ct i onal lyciui te distinct, there can be 
'some intei'play between 'format i \'e aiid summati^'e i-esearc!-"! . 

In general, ho\se\er, ^the format i ve -Research -itzenda should be distinct 
becaust';^of its *im^rove;?.ent ori cnt at i on / ti\at is-, its concent rafi on nn the 



ERLC 




domain of manipulable factors under the control of production. Of course, the 
production staff must be concerned witti the assessment of behavioral objcc- 
tivcs, but a final "report card" on hovv/.well the program performed may offer 
no insight whatsoever on how to improve j.t the next time around (if there ever 
is to be a next time). Many formative research questions will tend to resemble 
closely, the questions a good producer-director 'udulfl be naturally considering 
anyway in the design and production process e.g. / Is this enough refep:'ence 
to the previous programs for the students to make the logical connections to 
this program? Will the students--d6tect the subtle humor in the closing scene, 
or will that be confusing to them? Given a battery of such questions, it should 
be evident that, to be resppnsive, considerable methodological ingenuity and 
variety are required on the part of the.in-house formative research staff. It 
^ ^ecommanded that bot}i backQ^omd /le^^ecuich and (^on.mativz ^o^aoJich be condacJidd 
by Ivi-konAt /le-'ieaAc/t ^ta(^{^, .s^^vsitfivc. to £/ie pa/ittcuZai wecdi AiT and committtd 
to toYiQ'ttnm^ good n.^atA.OYiblvip^ i^j'itk AIT pe^^icnuicf . With formative research 
responsive to the real needs of production, production can be an enthusiastic 
supporter ^nd consumer of research. • 

Summative Research • ' ^ 

As defined earlier, summative research assesses the extent to v^fhich a 
♦ * 

-program (or series) has reached its objectives, Summative research probably 
resembles the, most common expectation of Ivhat research and evaluation is about. 
It is conducted because decision-makers need overall f)erformance data. Did 

\I spend my money wisely? Does this program merit additional funds for revision? 
Sh6.uld this program be scheduled for ne:>^t^year? These are decisions that need 



the summative research output. 



-.Typically, many of the consumers of summative research are outside^ the.^ - 

organization, A recommended policy is that atl ^ummativd KOMioAch ^OK quX^id^ 
CLon^imoM fae aondacXtd by /iQ,6Q,aAck Qn^oup^ ouuUldt AIT, {^oh, {leo^a^u o^' dei&c/aiid. 
obj(L(itl\)ittj and cAtdlbJXitij . It is further recommended that .;t/ie -de^vigia and 
mea6a/i/cng In^iAammt^ {^o^.alZ cohtAactdd 6immcC€ivt n.z6Q.cuick '6houZd 'bQ. apph.ovQ,d 

■ {^lut by' in-koiJU>d AIT adml\'iUtAcuto'nJ^ to Inlaid that all Koldvant objdCjtivQA 
oAd mdO^vAdd In an app^iopfUatd.manndn.. After A^IT has approved the research 
procedures, the outside research agency should be -autonomous ; 

Unfortunately, it is still too common to find situations where summative 
research is an afterthought. Researchers have had to resort to. monitoring 
a compileted program, inferring what the objectives or producer's intent must 
h^ve been, and then* writing summative evaluation items on that basis. This 

"should not happen. The planning of summative research should begin as soon, 
as the planning for the program begins. I'f there are to be pre- test/po3t-test 
^comparisons as paJti of the summative research; it is obvious that the research 
must be in the field before the first airing. ' ' 

There are arguments for incorporating standardized instruments into 
summative .research and arguments against. Supportive arguments include: 

a) You tKereby get a measuring instrument that has been field tested; 

b) You have a common basis on which the performance ''of other groups can be 
compared; and i . ' . ' 

c) There are national norms established for many-^of these instruments. ., 
Arguments against standardized instruments include: 

a) Many of them are not designed as a measure of program effects; 

b')' There is some possibility -that an edcisting instrument could influence the 
curriculum by tailoring the objectives to fit the instrument; and 

c) Most importantly, standardized 'tests may have minimal, overlap with the 



total configuration of program objectives. This hurts in two ways: for content", 
on the test but not covered in the program, there will be no demonstrable effect 
of the program. For content on the program but not on the test, significant ' 
effects could go /^measured. 

Tk^ Jtzcommzndcution aj> that o^g-inat MyUitH.wmnt-s 6e u6e.d to tvUiUAd pxzcJj^d 

Some obj^ectives are easier to measure than others, and there may well be 

understandable pressure to lean toward the more' readily fneasured outcomes in the 

program design. One can invoke the argument that if the objective can^t be 

specified, tlj^en it has, no ^p lace in the program or the research. Such a hard-nosedi 

^ * ' i y I 

■ . . ■ • ' . ■ • ' K i 

position coi^Ld limit the curriculum -goals to th'e capacity, of the art^afid science ,■ 
of measurement. For television^ which has a marvplous ability to communicate 
non-verbal infoirmgrtion and to involve people emotion^ally , this would be. an | 
unfortunate limitation. Reasonable men- and women should work out on a case-by- 1. 
, case . basis ttie best compromise between two desirable outcomes: rigor of measure-^ 
ment.and exploitation of the potential of the television medium. It might, for . 



example, be reasonable to require, behavioral measure^ for a^^majority of the -j 

^Jesired outcomes, with other objectives being allowed far less rigor. Even an / 

/ ■ — rr- : ■ / . . ' ^ • / 

occasional shot in the dark with no measurement whatsoever should not be precliided 

It should be noted that a greater ability on the piart of researchers to measur^ 

" . • i' 

the more, elusive outcomes (affective variables, for example) would facilitate 'I 
' their formal incorporatio^n into various curricula. If AfT is ever inclined, 
contrary to an earlier recoiMTiendation, to sponsor basic research, then such^ 
measurement research would be an excellent investment. , ^ I 



After recognizing and supporting completely the SQienti^c values of | 
^tgor, clarity, precision, reliability, vafidity, and. the like, one must alsL 
recognize that most research, even the best research, compromises downward from 
. the lofty ideal. With great resources some compromise will be necessary, arid 



) 



with limited resources, compromise is unavaidabl^^^. In decis i oii-oiM^iitcd 
research, one frequently is in the position cl- being able to increase the rigor 
of a study for a few thousand dollars more, but then deciding as a matter of 
priorities in resource allocation that the return is not Worth the investment. 
How far downward one can compromise artd still find utility in the data is a 
value judgment that should be made on a case-by-case basis. Again, the. operative 
criterion is uXi^Ltij in the decision-making process. Frequently, the options 
available are to make the decision \ith no data at all^ or to make the decision 
with^ \lata that are considerably compromised from the ideal. In such a situation 
it is not uncommon for reasonable people to prefer soinc data over no data at 
all. The research data combine wi th .whatever othe"^ formal and informal inputs 
the decision-maker can muster to infTcience the analysis and the conclusion. / 



Poli'cy Research 



Policy research, the last of the four categories of decision-oriented 
research to be covered here, will be easier to discuss if subdivided into two 
parts: (a) policy research tailored to the policy-making needs of AFT; and (b) 
policy research tailored to the. policy-making needs of larger societal and 
governmental units. 

The organizational mach:i'"ery requ.' red for on-<;oipg background research, 
recommended i^ ^arliex • ''.i^^ ' ' axs^^ oe**i..jal for serving 

AIT's pcii .aki~ needs. • ^ckgv- ^ . h , aata '^,ould help lay the 

groundwork intiVl ^ ^. ^-^ •^^nez.fic school lel' is^un materials. As 

xn-hovL ' poll -y re'- ■ .ni' couid h<.lp prcviJ'j •.)?./\s for the sus- 

t;ined 7ti'iew, ;0c fi cat loi. , and cration of fi[eneral A.."^ ol: .y. Tlie two uses 



PRir 



of 'research data are, of course, highly interrelated. The policy research does 

.not generate policy; it generates data that' top level admin-istrators should 

find us.eful irt policy areas . The^- non-exliaustive agenda suggested for background 

vresearch is equally well suited' for policy-making needs. . ^ ' 

In brief review, the suggested areas were assessment of needs , 'of "audi^encQ, 

variables (both demographic and psychological) , and of physical system variables 

(production, distribution^ reception), and feasibility studies for program ideas... 

At a .somewhat -higher level of ■ abstraction; these activi-ties can be thought of 

as a form of sustained system mor^itoring combined with, predictive simulation 

models. System m.onitoring should yield multi-faceted status reports on the 

\ ) 

system in which AIT operates. In program planning/ predictive simulation models 
should rbring to bear logic and evidence in predicting the consequences of spending 
resources before .actually committing the resources.. In policy deliberations, 
predictive simulation, models .should bring to ^b^r logic and evidence in predicting 
the future status of* the system. I * \ ' . ' — r 

Consider several hypothetical (but, it Is hoped, plausible) policy issues. 
The value of research-r^input should be self-evident'. 

"To what ex^nt should AIT diversi fy' its media products dVer the next 
fifteen years?" Predictions on the future status of the system would obviously ■ 
be required here for the decision-making* process, "How will the needs of the 
school systems change?" "How will technological developments affect AIT?" And 
so forth. . ^ 

"What are the major factors, the convergence of which should cause AIT to ^ 
undertake production activity and the non- convergence of which should preclude AIT 
revolvement?" Responsibility foT 'such blockbuster ^policy issues resides with ^ 



the- admin: strative deci sion-jTiaker , but iic or she sliould hul it most iisl^I:\i1 " 
to be able to cIVcck out cmpi ri ca 1 1 >' at least some of the jiilu t i tude of 
assumptions that Feed into such d : I'i be rat i ons . * 

The research staff and the ac'mi ni str!itor should cooperate in hr'/akim^ down 
the big polic)- questions int6 relevant ques t i ons .that ■ can he answered. The 
question "What should a rational person know before settii.g policy nn this 
complicated issue?" should geneij-ate -a host of items, .som? of which r"ui , and 
some of wh i cli cannot , reasonably be assigned to research. 

The policy needs of societal units that involve AIT activities arc anotlier 

matter. AIT should 5c a cooperative participant in contrihutirg whatever it 

can to these other "outside"- policy issues coming from responsi Ue agencies, hut 

would not actually conduct: the research. Presumably, AlT woul have a clear 

position and would servtT^^^^ir/ a'uvocate', for example, in government consideration^ 
V t ^ \ . ^ j 

of the role of educational fifiedia fbr the next decade. In another policv setting, 

AIT could sliare releVant experience as an expert \;itness (as, for example, in 

some state's deliberations about the pro? ^and cons of sex education in the 

schools, or emotional health in the S(^iooIs). In another setting, AIJ ma>' simply 

provide data, as for example, in response to a UNESCO sij^rvcy. * 

* * ' 

Because of the impact of television and its multitude of effects (and side 

effects) some of which are considered positive and i^ome negative, some of- 

* 

which are inten^ded and some unintended, some controllable , some uncontrollable -- 

the AIT areas of activity will inevitably brush against policy issues in a ^ 

variety of other-^agencies^ and organizations. These encounters offer, opportunities 

for positive influence in a rather wide^spectrum of policy. 

. \ ■ 
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• ^ Section* III: Pol'icy Recommendations * o"-^ 

I have tried to broaden the typical conception of/ the role of research 
and evaluation in televised instruction to include research inputs ^ a variety- 
of points in the system. By going under the^rubric^ of decision-oriented 
research, a, variety of other elements tend fo flow logically from such 'a 
. starting point: identification of research problems, the ch.aT(nels of coramunita- 
tion, the chain of »cdmmand> the consumer of ■ reseaircih reports , the delineation 
of meaningful distinctions among ^ypes of rese^^£]r*tod evaluation, and the 
Specification of research obj.ectives, to list a few. ' • 

. . ■ ■ ^ . " • : .. . • . ' •■ ~ ■ H " , 

Research methodology was not considered at all, and a variety o^ termino- 

. \: - ■ '■ ' 

logical issues and .territorial disputes were-'simply bypassed and left for another 
day,. The purpose of the paper was to review'/ discuss"* the areas in which research 
and evaluation could make positive contributions jto AIT, and to discuss some of 
the policy issues that relate to th^Dse potential/ contributions. For all such 

y ■ ■. A. ' ■ ' . ■ ■ ' 

^ issues, there is a recommended position fox your consideration and discussion. 
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In isolated form, here is a summary list of research polity recommendations,' 



most of which were covered in the preceding text* 



0 . 



1, 



Each^MT oicXl\)ity ^kouJid havz a /li^zaidi and zvaluaZ'Lon compondnt. 

2. AIT ^'houZd n,Q^tnxQZ It^ n,2Azcuidi Q.{^{^oKt^ to 'ddCyU^on-'O^Xfiivttd cl\'£^ 
p/Lodact-zSpeci^^c./Le^eo/Lc/t and zvaliicvtLoyi. <AI'T should cooperate with but ^ 
not sponsor basic or theoretical research. If, ..for whatever reasons, AIT . 



doii6 decide to\:i,nvest in basic research, this research should be in areas 
that have high probability of practical pay-off, such as in new measure- 
ment methods- to index non-cognitive effects. <4 • 

.3. Backgn^ound Kuza/idk and ^n-hoiUiZ pdtLaij Kz6zaJiah ^(louZd be. 

^ ^-^^ * . . " ' . ;. , 

^tabLUhzd 06 a pzmantyvt and^tntzg^at pant AIT, t/iayii^cmding spzcLlf^tc: 

. ■( , ■ ^ ' . 

4. FomG:tivz KQ^za^cJi ^hoiM be aondUati^d by an In-hdu^z KUzoJick 

6ta^, iA)on,klng tn ^thz ^2Avtcz 'o{^ pJioductyLon, and oA^gai^tzcutio natty ^uooAxUnatz 

. ' ^ ■ ' ■ 

to pH.oduction* A prerequisite to appointing production decipion-makers 

should, be their willingness to work with formative research in the improve- 

ment of the product. . * 

5. X)n(iz%IT Ui' dortmcttzd to a pKoduoJxon jo^ojzct, top pKlonJjtij {^ok 
n.2J;,(i.aA(ik n^uou/icz^ ^hoiitd go to ionxmXLvz KUzaficix. 



6. ThoJiz should always be <6ome loHm '^umi(r\ativz KUzoKck, bat {^dnding 
hoAz ^ 6Z(iondan.y to* a quxiLLty {^onjinativz /le^ea/ic/t pwgKotn. ^ 

7. kVi AQAzanxLk and zvaZaatton ^ to be judged by tiiz QMltQJu.on 
iitltitijl tn thz vaJUou6 d^zcA^ton^maklng p^toc2A^(U. This utility will be 
enhanced if the consumer can be 'involved in setting, the specifications for 



the research and evaluation projects. This holds , for all four catego'rie.s 

■ . ■ ' - ' • X • - ■ / 

of research discussed in this paper. ' ■ / 

■ ■ ' / 

8. SmmcutivQ, /iU2jcUich {^on, ouut&^do, coemption 6kouM be (U6Z^nQ.d on 

CGiWuict to ouut6idji, dompoXdYvt, A,2At(VLcih g/ioap^. Control over the/design 

of the summative research and the instruments used should remain in-house 

,1 ' l! ■ ' •*.* 

with AIT^ but once approved, the outside group should be autonomous. 

// 

1^ ' ■ ■ ■ ■ , 

. ■ . ■ .'^ . 

tixak itayidctAdlztd t^^t/^ 6hoaZd fae mployad io^ ^oAmcutivn and ^dinmcitlvo, KQ^aa/idk. 

' ■ ' ;/ ■ ^ ■ ■ 

// 
jj 

10^ ^^^on,t Iniomatlon oif^ldoAM ^oZoxUiiL ok qubtt {^KOm /i^eo/tc/i {^ind^ng6^ 

'I * 

{^on, pubtiaity puApo6U [no mcuttoA wkdthoA baakg^oimd^ {^omativz, j>ummcutivQ,, 

/ . 

OK poticy KQj^ta/ick >U AJivolvQ.d]^ a KoMpomlhlz KoMtcvtok o^^ilcvi should 

. '• . • . . ■ ■ ■'/ . ■ 

dppKovt iht KoZoxUiH to tn^uAQ,^ that the, tntoApKztatton b^lng cjywayad aj^ tn 

■ , ■ i 
^a it conAtct and 6upppAte.d by tkt data, - - / 
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